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The  Curvature  PrimalSketch 


Ilaruo  Asada  and  Mir  had  Drady 


Abstract.  In  this  paper  we  introduce  a  novel  representation  of  the  significant 
changes  in  curvature  along  the  bounding  contour  of  planar  shape.  We  call  the 
representation  the  curvature  primal  sketch.  We  describe  an  implemented  algorithm 
that  computes  the  curvature  primal  sketch  and  illustrate  its  performance  on  a 
set  of  tool  shapes.  The  curvature  primal  sketch  derives  its  name  from  the  dorse 
analogy  to  the  primal  sketch  representation  advocated  by  Marr  for  describing 
significant  intensity  changes.  We  define  a  set  of  primitive  parameterized  curvature 
discontinuities,  and  derive  expressions  for  their  convolutions  with  the  first  and 
second  derivatives  of  a  Gaussian.  The  convolved  primitives,  sorted  according  to  the 
scale  at  which  they  are  detected,  provide  us  with  a  multi-scaled  interpretation  of 
the  contour  of  a  shape. 
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1 .  Introduction 


R<<  rn!  I>  'Hr. ids  I  tlS'Ja  HIS'-’I).  I9S.1.  |(M;ti  Mniily  .mil  AmhI.i  I'M. 
ha\r  ill  1 1  ml  IK  cd  .1  i  t’|>r«“-iMit  ,il  mu  i)l  I’ao  ililiicfiMon.il  >hape  called  .•  t! 

si; mint  !  ri<  i  (M.S )  I  lie  ; y.riunrl  ri<  as pci  I  -  of  1  lie  |  1 1- j 1 1  n t :» t  ion  an  -ii  iniiia  i  m  <1  in 

Figure  I.  I  igurc  la  slums  ilial  both  I  lie  l>v>  u  ml  i  n  o  contour  and  the  cm  limed  region 
of  a  shape  arc  represented,  lor  reasons  discussed  in  jllrady  and  .V-adn  Ill's  In  I  mine 
11)  shows  ihc  hieraichii  al  decomposition  ol  a  shape  into  mi  holt  j  «•»  I  >■  >iuoot  lied 
local  symmetries  arc  being  applied  to  recognition  and  ins|icclion.  ami  uitc'iaicd 
u  it  h  a  s  y  s  t  tiii  that  i  an  reason  by  analogy  [Winston  I  UNO.  Winston.  Hinlord.  ha!/, 
and  Lowry  IllNl]  to  investigate  the  relationship  between  ('unction  and  form  llr.uh 
I984bj. 

The  representation  ol  the  bounding  contour  ol  a  shape,  particularly  of  the 
significant  changes  of  curvature,  is  railed  the  curvature  primal  sketch,  and  it  is 
the  subject  of  this  paper.  There  arc  three  reasons  for  discovering  and  representing 
significant  curvature  changes.  First,  they  amount  to  a  rich,  stable,  description 
with  local  support  that  can  enable  the  recognition  of  partially  occluded  objects. 
Second,  they  provide  a  set  of  knot  points  for  constructing  a  perceptually  close 
spline  approximation  to  a  contour.  Finally,  it  is  possible  to  describe  a  shape  at 
multiple  scales  by  interpreting  the  curvature  events  along  the  contour  at  various 
scales.  The  basic  idea  of  the  curvature  primal  sketch  first  appeared  in  [Brady  and 
Asada  1984). 

We  call  our  representation  the  curvature  primal  sketch,  because  the  repre¬ 
sentation  of  significant  changes  in  curvature  is  analogous  to  the  primal  sketch 
representation  of  intensity  changes  in  grey  level  images  [Marr  1974,  1976].  For 
example,  a  discontinuity  in  the  orientation  of  a  tangent  to  the  contour  (Cl  discon¬ 
tinuity)  is  perceived  as  a  corner  and  is  analogous  to  step  changes  in  intensity.  A 
discontinuity  in  curvature  (C2  discontinuity)  is  a  smooth  junction  of  two  pieces  of 
contour  and  is  analogous  to  a  gradient  edge. 

Our  approach  follows  that  of  Marr  [1974],  We  define  a  parameterized  set  of 
idealized  curvature  changes.  The  set  includes  composite  structures  such  as  end 
and  crank  (sec  Figures  8  and  9)  which  arc  analogous  to  thin  bar  and  line  ending 
respectively.  We  then  derive  expressions  for  the  convolution  of  the  primitives  with 
the  first  and  second  derivatives  of  a  Gaussian.  The  instances  are  described  at  a 
variety  of  scales  in  terms  of  the  positions  of  the  local  maxima  of  the  derivatives. 

We  use  the  idea  of  multiple  scales  in  two  ways.  First,  it  is  possible  to  find 
an  instance  of  a  curvature  change  primitive  at  one  scale,  even  when  it  cannot 
be  found  at  other  scales.  There  are  two  main  reasons  for  this.  First.,  the  filtered 
responses  of  two  nearby  curvature  changes  may  be  confounded  at  larger  scales,  and 
second,  instances  at  smaller  scales  may  be  caused  by  noise.  A  pertinent  observation, 
concerning  intensity  changes,  but  relevant  because  of  the  analogy  between  curvature 
and  intensity  changes,  was  made  by  Marr  and  Hildreth  [1980]  and  Hinford  [1981], 
and  discussed  more  explicitly  by  Canny  [1988).  It  is  that  there  is  an  uncertainty 
principle  between  the  detectability  of  an  event  and  its  accurate  localization  in  the 
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Figure  1.  a.  A  portion  of  a  shape  and  the  corresponding  smoothed  local  symmetry 
representation.  The  solid  lines  form  the  curvature  primal  sketch  representation  of  the  contour. 
The  dotted  lines  form  a  representation  of  the  region  enclosed  by  the  shape  fragment,  b.  The 
representation  of  a  hammer,  showing  the  subparts.  Other  links  form  a  ISA  hierarchy,  embody 
constraints,  and  detail  the  function  of  the  shape.  For  details  of  smoothed  local  symmetries,  see 
[Itrady  and  Asada  198-1]. 

presence  of  noise.  That  is,  the  coarser  the  edge  finding  operator,  the  more  the 
signal  to  noise  ratio  (a  measure  of  detectability  of  the  event)  is  improved,  while  the 
smaller  the  operator,  the  more  accurate  the  localization.  Bv  varying  the  width  of 
an  operator  it  is  possible  to  vary  the  trade-off  in  signal  to  noise  versus  localization 
[Canny  10«3].  Larger  scales  can  be  used  for  detecting  the  occurrence  of  events  and 
smaller  scales  for  localizing  the  events  accurately. 

We  also  use  multiple  scales  in  a  second  way,  namely  to  build  a  multi  scaled 
description  of  a  shape.  The  idea  of  convolving  a  signal  with  multiple  operators 
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;il  <i  i  (It- re  it  l  spatial  scales  hern  explored  lor  several  year-  b’crc-iit  Iv  .  Ikhuvit. 
Wilkin  IHVt  lias  [ironiot  til  l  lie  uli  a  <>|  scale  ,-inict .  .1  representation  ol  .1  •>  1  ••  1 1 ;« I 
lli.il  (oii'-i'i-t  o|  its  ron  volu  t  ion  ;it  multiple  scales,  ,\n  import. mt  port  ol  W  if  in  - 
prnpo~.il  is  t  hr  .111 1  oiimt  ic  del  erminal  ion  ol  the  set  ol'si.ilrs  .it  which  it  i~  u-«|  ,i  1 
to  describe  .1  sijiii.il  s\  111 1  >  i  >  I  i  1  .ills  Wilkins  scale  spine  rcpi  1sri1t.1t  ion  1-  .1  iciiiaiv 
tree  ol  /rro  crossings.  Hr  did  not  ultrmpt  to  mterpnt  tlir  mull  1  p li  descriptions  in 
trims  ol  primitive  events  as  vve  have  done. 

'ill. Hr  and  Poggio  iltls. {.1]  have  provided  some  tlirorctir.il  underpinning  lor  I  lie 
scale  space'  representation.  I  liey  have  shown  that  the  contour  ol  zero  crossings  «>) 
second  derivatives  (  'hugcrpnnt  1  mat  pieservc  enough  111I01  mat  ion  to  reconst  iiirt 
the  original  signal  to  within  a  constant  scale  factor.  They  also  -how  Auille  and 
Poggio  IHS3b]  that  a  Gaussian  lilter  is  essentiallv  tinicpie  in  having  the  propertv 
that  /.(To  crossings  are  not  introduced  as  one  moves  to  coarser  scales.  However,  as 
we  discuss  further  hclovv.  this  mathemtilical  property  cannot  be  relied  on  in  an 
implementation  of  the  interpretation  process,  since  a  discrete  set  of  scales  may  make 
a  match  ambiguous  when  the  responses  of  two  nearby  <  \\  nts  are  confounded.  To  our 
knowledge,  the  curvature  primal  sketch  reported  here  is  the  first  implementation  of 
an  interpretation  process  that  generates  multiple  scale  descriptions  from  multiple 
convolutions. 

An  instance  of  a  curvature  change  primitive  that  is  detectable  at  a  large 
scale  suggests  that  it  is  a  dominant  feature  of  the  shape.  One  which  is  inhibited 
by  a  dominant  event  at  a  large  scale  and  can  be  detected  only  at  smaller  scales 
indicates  a  feature  that  is  geometrically  less  significant.  However,  geometrically  less 
significant  events  arc  not  necessarily  less  important  for  the  successful  execution  of 
a  vision  task.  A  small  dent  in  an  object,  for  example,  may  be  considered  noise  when 
the  task  is  to  identify  the  object,  but  it  may  be  crucial  when  the  task  is  to  inspect 
for  defects. 

In  section  2,  we  summarize  the  idea  of  the  curvature  primal  sketch  and  define  a 
set  of  primitives  and  their  instances  in  scale  space.  Section  3  discusses  the  algorithm 
to  build  a  multi-scale  description.  The  results  in  the  experiment  on  a  set  of  tools 
arc  shown  in  Section  4. 


2.  Represerting Significart  Changesln  Curvature 

The  contour  of  an  object  is  represented  by  a  spline  whose  knots  are  the 
significant  changes  in  curvature.  The  smooth  curves  between  the  knots,  what 
Perkins  [l 978]  calls  concurvcs,  can  be  parameterized  by  approximating  the  portion 
of  the  contour  linearly,  with  quadratics,  cuhics,  Cornu  spirals,  or  any  other 
suitable  function.  Here  we  are  more  concerned  with  the  problem  of  finding  knots 
corresponding  to  curvature  discontinuities  than  the  parameterization  of  smooth 
portions  of  contour  between  knots.  The  detection  and  localization  of  discontinuities 
is  more  crucial  because  the  good  parameterization  of  a  piece  of  contour  depends 
on  them  (Figure  2). 
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Figure  2.  Knot  points  should  mark  signiliranl  changes  of  rurv;il  lire  along  the  contour.  a. 
A  corner  fragment  on  a  contour,  b.  Approximation  when  knot  points  (marked  by  crosses)  are 
placed  to  either  side  of  the  perceived  corner,  c.  Approximation  when  the  corner  is  made  explicit. 
(Weproduced  from  Ihrady  and  Asada  IflHd,  Figure  8j) 

In  our  experiments,  piecewise  circular  approximation  of  the  contour  has  proven 
to  be  good  enough,  provided  we  can  first  accurately  locate  all  the  semantically 
significant  curvature  changes  as  knots.  In  Figure  3a  we  show  the  contours  of  a 
number  of  tool  shapes  found  by  the  Canny  edge  detector.  Figure  3b  shows  the 
circular  spline  approximations  to  those  contours  once  the  curvature  discontinuities 
have  been  located  by  the  method  described  in  this  paper.  Figure  3c  shows  the 
overlay  of  Figure  3b  on  Figure  3a. 

We  propose  a  particular  set  of  primitive  curvature  changes.  We  analyze  each 
primitive  at  several  scales  using  an  ideal  parameterized  version  /,•(«)  of  the  primitive, 
where  s  denotes  arc  length  along  the  contour.  We  then  determine  analytic  forms  for 
the  convolutions  of  /,(s)  with  the  first  and  second  derivatives  of  Gaussian,  namely 
G'„  *  fi  and  G"  *  fr,  where  G„  denotes  the  Gaussian  of  standard  deviation  a: 


G„(t)  =  (I/\/27TO')exp(— i"/2<r2). 

Equivalently,  by  the  derivative  theorem  of  convolutions,  we  determine  the  analytic 
forms  of  ( G„  *  and  (G„  *  fi{s))".  Then  we  construct  a  program  to  find 

instances  of  the  curvature  change  models  along  actual  contours. 

2.1.  Two  Basic  Discontinuities 

We  bc^in  by  deriving  expressions  for  the  convolutions  of  the  two  basic  models, 
the  corner  and  the  smooth  join,  each  of  which  have  a  single  discontinuity.  The 
convolutions  of  the  compound  models  namely,  crank,  end,  bump,  and  dent,  are 
formed  by  .appropriate  superpositions  of  the  filtered  responses  of  these  two  models. 
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Figure  3.  Illuslr.if  ing  the  adequacy  of  circular  splines  for  a  s»  t  of  tools.  As  discussed  in  Brady 
and  Asada  [1981],  a  circle  may  he  replaced  by  a  straight  line  depending  on  the  condition  number 
of  tin’  best  fitting  circle  equations,  a.  The  original  contour,  b.  The  circular  spline  approximation 
after  finding  the  significant  changes  of  curvature,  c.  Overlay  of  b  on  a. 

1)  Filtered  response  of  a  corner 

Figure  4a  shows  a  corner  fragment  consisting  of  two  circular  fragments  of 
curvatures  /c i  and  k<),  enclosing  an  angular  discontinuity  of  <j>.  Figure  lb  shows  the 
corner  model  in  orientation  space,  relating  the  orientation  of  the  tangent  to  the 
curve  to  arc  length  along  the  curve.  The  corner  model  is  defined  by: 


tejs  +  c 

+  C  + 


0 


if  s  <  0; 
if  s  >  0. 


As  we  shall  see,  the  convolutions  of  the  model  assume  particularly  simple  forms  when 
the  curvatures  of  the  circular  arcs  flanking  the  corner  are  the  same,  acj  =  k.2(=  /c) 
say.  We  call  that  special  case  a  pure  corner.  Its  analytic  form  is: 


KS  -f  C 
ItS  -f  C  + 


if  s  <  0; 
if  s  >  0. 


We  now  convolve  /ror„rr  with  the  Gaussian  G„(t),  and  find: 
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This  has  a  particularly  simple  form  for  a  pure  corner  namely: 

•) 

\f2-Ko{Ga  *  f,,r)'{s)  =  0exp(— ~-2)  +  \Ph\KO . 

The  function  (C„  *  is  shown  in  Figure  lc.  It  has  the  shape  of  a  Gaussian 

that  is  offset  from  the  line  0  =  0  by  the  scale  and  curvature  dependent  constant 
y/2nKO  and  is  attenuated  proportional  to  the  angle  <f>. 

Similarly,  convolving  with  the  second  derivative  of  a  Gaussian  (approximately 
equal  to  a  diffcrence-of-Gaussians)  yields: 


_ ±  ^2  2 

*  frorncr)  (s)  =  rf  S  CXp(  —  ~ — 5  )  •+•  (/C^  —  /C|  )  CXp( - n  ) 

O  iO c  2 Oi 

This  function  is  shown  in  Figure  4d.  It  has  a  zero  crossing  near  the  corner  separating 
two  peaks  of  opposite  sign.  In  the  case  of  a  pure  corner,  the  expression  assumes  the 
following  simple  form: 

'Jz*o{G„  *  fpr)"{s)  =  —f  sex p(- ~). 

o-  2<t‘j 

The  distance  along  the  s-axis  between  the  peaks  is 

.  _ p2  L  v,  44>- 
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In  the  case  of  a  pure  corner,  this  expression  reduces  to 
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Figure  4.  a.  A  corner  fragment  consisting  of  two  circular  fragments  of  curvatures  k  ,  and 
«'j,  enclosing  an  angular  discontinuity  of  <i>.  I>.  The  corner  model  in  orientation  spare,  relating  the 
orientation  of  the  tangent  to  the  curve  to  arc  length  along  the  curve.  In  the  rase  of  a  pure  corner, 
the  slopes  k |  and  k>  are  equal,  c.  The  function  ((!„  *  It  has  the  shape  of  a  Gaussian 

that  is  offset  by  the  curvature  k  and  is  attenuated  proportional  to  the  angle  <t>.  d.  The  function 
((!„  •  which  corresponds  to  convolving  the  corner  model  with  the  second  differential 

of  a  Gaussian.  It  has  a  zero  crossing  near  tfic  corner  separating  two  peaks  of  opposite  sign. 

Note  that  dpr  is  independent  of  <j>  and  the  pair  of  peaks  merges  when  the  scale  gets 
smaller. 

The  formula  for  the  heights  of  the  side  lobes  is  complex.  In  the  special  case 
*i  —  k'2,  they  are  both  equal  to 
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2)  Filtered  response  of  a  smooth  join 

The  corner  model  fCOTnrr  and  its  first  derivative  arc  discontinuous  at  the  origin. 
If  <t>  is  set  equal  to  zero,  the  orientation  of  the  tangent  is  continuous,  but  the 
curvature  is  not.  We  call  such  a  curvature  change  a  smooth  join.  Figure  5a  shows 
two  instances  of  the  model,  whose  defining  equation  is 
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Figure  5.  a.  Two  instances  of  a  smooth  join,  in  which  the  signs  of  the  Hanking  curvatures  art 
the  same  (top)  and  opposite  (bottom),  h.  The  smooth  join  in  orientation  space,  c  Tin  tillered 
response  of  a  smooth  join  to  the  first  derivative  of  a  Gaussian,  d.  The  filtered  response  of  a 
smooth  join  to  the  first  derivative  of  a  Gaussian. 


When  the  signs  of  the  curvatures  flanking  the  smooth  join  are  opposite,  the  smooth 
join  is  called  an  inflection.  Figure  5b  shows  the  smooth  join  model  in  orientation 
space.  It  is  well  known  that  smooth  joins  are  difficult  to  perceive  unless  there  is 
a  large  difference  (/C|  —  acj)  in  the  flanking  curvatures.  For  example,  l  liman  [1976] 
and  Brady,  Grimson,  and  Langridge  [1980]  have  investigated  smooth  curves  that 
have  a  single  C>  discontinuity  in  curvature.  Figure  6  shows  a  curve  that  is  the 
smooth  join  of  two  circles.  The  curve  and  its  tangent,  are  continuous,  hut  there  is 
a  step  change  in  the  curvature  at  a  point  along  the  curve.  Evidently  the  curvature 
discontinuity  is  not  perceived. 

Inserting  =  0  into  the  equation  for  the  response  of  a  corner  to  the  first 
derivative  of  a  Gaussian,  we  find 


V2no{Ga  *  —  kl.  [  cxp(—  -  [,  )dt  +  /ci  /  exp(  -  \ \  )dt. 
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Suppose,  w.  hout  loss  of  generality,  that  k\  <  «■„>.  Then 
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'This  function  is  shown  in  Figure  or.  It  has  the  form  ol  ;i  smooth  step  v .  h-isc 
height  is  proportional  to  (rci  —  k  j  ) ,  the  dillerome  in  curvatures  Hanking  the  join  II 
the  dilierenec  in  flanking  curvatures  is  small  the  sirp  nay  not  he  perceived. 

Differentiating  again,  we  find  the  filtered  response  of  the  smooth  join  to  a 
second  differential  of  a  Gaussian. 


V2na{G„  *  /„;)"(«)  =  (/c2  -  /C|)exp(-~-2-). 

This  response  is  shown  in  Figure  5d.  It  has  a  single  peak  whose  height  is 
proportional  to  («;■_>  —  K\)ja.  Introducing  the  quantity  0,v  —  (k_>  -  K\)cr\Je,  we  can 
write  an  expression  for  the  height,  of  the  response  peak  that  is  similar  to  the 
expression  for  the  height  of  the  peaks  in  the  corner  response,  namely 

hgj  —  Ifgj \/o2y/2ne. 

The  detectability  of  a  smooth  join  is  essentially  determined  by  the  peak  height 
h^,  that  is  by  the  value  of  0VJ .  In  particular,  a  smooth  join  is  hard  to  detect  unless 
the  product  of  o  and  (k_>  —  k.j)  is  sufficiently  large.  This  is  consistent  with  human 
perception  of  changes  such  as  those  indicated  in  Figure  6.  It  is  not  known  how  the 
threshold  on  0,v  required  to  perceive  a  smooth  join  is  related  to  that  on  0  required 
to  perceive  a  corner. 

2.2.  Primitive  curvature  changes  and  their  detection 

In  this  section,  we  introduce  the  five  primitive  curvature  changes  that,  we  use 
in  the  curvature  primal  sketch.  We  call  them  corners,  smooth  joins,  cranks,  ends, 
and  bumps  or  dents.  We  define  each  primitive  and  show  it  in  orientation  space.  We 
show  example  responses  at  different  scales  when  the  primitive  is  filtered  with  the 
second  derivative  of  a  Gaussian.  Corners  and  smooth  joins  were  analysed  in  the 
previous  section;  in  this  section  we  show  different  scale  responses  and  give  detection 
criteria. 
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Figure  7.  The  responses  to  the  corner  shown  in  Figure  -1  to  at  several  spatial  scales  a. 

The  corner  primitive 

As  shown  in  Figure  4d,  a  corner  gives  rise  to  a  pair  of  peaks  whose  separation 
is  dcomcr •  An  expression  hj*.  for  the  heights  of  the  peaks  was  given  for  the  case  of 
a  pure  corner.  If  the  flanking  curvatures  K\  and  «•>  are  not  equal,  so  that  there 
are  simultaneous  C\  and  C-i  discontinuities,  the  zero  crossing  in  ( G"„  *  /r0rnrr)(s) 
is  not  exactly  at  the  origin.  In  practice,  the  difference  —  «2  >s  small,  so  that 
dcomcr  ^  dpc .  Note  that  dcorner  is  linearly  dependent  on  the  scale  constant  a,  and 
hence  it  monotonically  decreases  with  a.  This  provides  a  strong  clue  for  recognition 
of  corners  and  enables  spatially  close  events  to  be  decomposed  in  order  to  distinguish 
a  pair  of  smooth  joins  from  a  corner.  Figure  7  shows  the  second  derivative  filtered 
responses  to  a  corner  at  several  scales. 

The  smooth  join  primitive 

In  the  previous  section,  we  showed  that  the  response  of  a  smooth  join  to  the 
first  derivative  of  a  Gaussian  is  a  smooth  step,  and  that  to  the  second  derivative 
it  is  a  peak.  The  peak  height  is  (k<>  —  K.\)/(ay/2n).  Recall  that  only  when  the 
curvatures  flanking  the  smooth  junction  are  very  different  can  this  be  reliably 
fbund.  Inflections  produce  a  zero  crossing  in  the  first  derivative  response. 

The  end  primitive 

An  end  consists  of  two  corners  with  angle  changes  of  the  same  sign  (Figures 
8a  and  8b).  It  is  not  necessary  that  the  contour  fragments  comprising  the  end 
arc  straight  or  parallel,  nor  are  the  corners  required  to  be  right  angles.  An  end  is 
analogous  to  the  primal  sketch  intensity  change  called  a  “line  ending”  [Marr  1976]. 
Let  a  be  tl  °  distance  along  the  contour  between  the  corners  forming  the  end.  At 
large  (dimensionless)  scales,  when  the  ratio  a/a  is  greater  than  one,  the  filtered 
responses  of  the  contributing  corners  arc  confounded,  and  the  end  is  perceived  as 
a  corner.  At  smaller  scales,  when  the  ratio  a/a  is  less  than  a  half,  the  individual 
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Figure  8.  ii.  A  typical  end  iis  it  appears  on  a  contour,  b.  The  end  in  orientation  span 
the  second  derivative  filtered  responses  to  an  end  at  several  (dimensionless)  scales. 


corner  responses  are  separated,  enabling  the  detection  of  the  corner  pair.  This 
occurs,  for  example,  in  the  interpretation  of  end\  for  the  punch  shape  shown  in 
Figure  11.  Figure  8c  shows  the  second  derivative  filtered  responses  to  an  end  at 
several  (dimensionless)  scales. 

The  crank  primitive 

A  crank  is  like  an  end,  except  that  the  changes  in  angle  are  of  opposite  sign 
(Figures  9a  and  9b).  It  is  analogous  to  the  primal  sketch  intensity  change  called  a 
‘  thin  bar”  [Marr  1976].  Again  let  a  be  the  arc  length  between  the  corners  forming 
the  crank.  When  the  ratio  a  fa  is  less  than  one  half,  the  crank  is  essentially  signalled 
by  two  independent  corner  responses.  When  a  fa  is  greater  than  one,  a  crank 
produces  a  strong  central  peak  with  two  side  peaks  of  opposite  sign  that  are  at  most 
half  the  height  of  the  central  peak  if  curvature  differences  are  negligible.  Figure 
9c  shows  second  derivative  filtered  responses  to  a  crank  at  several  (dimensionless) 
scales. 

The  bump  and  dent  primitives 

Instances  of  a  bump  and  dent  are  shown  in  Figure  10a  and  Figure  10b.  They 
arc  typically  spatially  localized  and  correspond  to  two  nearby  cranks  of  opposite 
sign.  Figure  10c  shows  filtered  responses  at  several  scales.  They  are  reliably  detected 
by  finding  the  sequence  of  four  peaks  at  a  suitably  small  scale. 

Different  instances  of  the  primitives  at  suitably  coarse  scales  can  be  ambiguous. 
Figure  1 1  shows  several  ambiguous  contour  fragments.  An  example  of  the  ambiguity 


Figure  9.  a.  A  typical  crank  as  it  appears  on  a  contour.  b.  The  crank  in  orientation  space 
c.  the  second  derivative  filtered  responses  to  an  crank  at  several  (dimensionless)  scales. 


smooth  join  - 

bump/  dent 

Figure  11.  A  matrix  or  contour  fragments  that  appear  ambiguous  at  suitable  scales. 

between  4  crank  and  a  smooth  join  will  be  found  in  the  curvature  primal  sketch  of 
a  screwdriver  in  Figure  19. 
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Figure  10.  a.  A  typical  bump  as  it  appears  on  a  contour,  b.  The  bump  in  orientation 
space,  c.  the  second  derivative  filtered  responses  to  a  bump  at  several  (dimensionless)  scales. 

3.  Multiscaldnterpretation 

In  this  section  we  describe  an  implemented  algorithm  that  produces  a  multi-scale 
curvature  primal  sketch  representation  of  a  contour. 

Step  1:  Convolving  orientation  with  derivatives  of  Gaussians 

The  bounding  contour  of  a  shape  is  found  using  the  edge  finder  developed  by 
Canny  [1983].  The  filtered  responses  G'„  *  c(s)  and  G"  *  c(s)  are  computed  for  the 
contour  c(s)  at  a  variety  of  scales  a.  The  result  is  a  set  of  one  dimensional  arrays 
for  both  G'a  and  C".  Figures  12  through  15  show  examples  for  four  tool  shapes. 

Step  2:  Matching  the  Locations  of  Peaks  among  Scales 

The  locations  of  local  positive  maxima  and  local  negative  minima  in  the 
filter  responses  computed  in  step  1  arc  extracted  and  matched  among  the  scales, 
producing  a  tree.  Ideally,  the  tree  is  equivalent  to  a  “fingerprint”  [Yuille  and 
Poggio  1983].  However,  we  cannot  trust  the  tree  entirely  because  the  match  can  e 
ambiguous  for  two  nearby  events.  For  example,  each  of  the  two  peaks  that  signal 
an  end  splits  into  a  pair  of  peaks  when  the  ratio  a /a  is  one.  However,  due  to  the 
discretization  of  the  scale  factor  and  the  efTect  of  the  curvature  difference  between 
two  fragments  of  the  contour,  it  is  possible  that  only  one  of  the  peaks  splits  into 
two  at  a  particular  scale.  In  such  cases,  the  match  between  the  new  peak  and  the 
old  ones  is  ambiguous.  We  need  knowledge  of  the  primitive  type  in  order  to  build 
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a  complete  tree,  hut  such  knowledge  is  not  available  until  the  tree  i-  interpreted 
C'onsei|iienl Iv,  it  is  necessary  to  leave  the  tree  incomplete  and  take  it  into  account 
at  the  interpretation  stage. 

Step  3:  Parsing  the  tree 

The  parsing  method  we  use  is  simple.  In  the  early  stages  of  this  re-can  h,  we 
considered  using  heuristic  methods,  such  as  relaxation  and  dynamic  programming, 
in  order  to  cope  wit  It  the  ambigu  it  y  that  occurs  at  t  he  interpret  at  ion  of  t  he  responses 
of  two  closely  located  events.  However,  it  has  turned  out  that  in  the  vast  majority 
of  cases  these  responses  can  be  decomposed  easily  by  looking  at  the  movement  of 
the  peaks  over  several  scales  and  taking  account  of  the  convergence  property  of  a 
corner  response.  This  experience  supports  idea  of  scale  space  as  a  representation 
technique. 

Instances  of  the  composite  structures,  namely  ends,  bumps  and  cranks  arc 
searched  for  first.  The  movement  of  the  peaks  at  lower  scales  and  the  heights  of  the 
peaks  arc  the  only  clues  for  the  detection.  Once  an  instance  is  detected,  the  tree 
is  refined.  The  search  proceeds  to  lower  scales  until  the.  scales  are  exhausted.  Next, 
corners  are  searched  for  in  the  same  manner  and  finally  inflections  and  smooth 
junctions. 

Step  4:  Computing  Knot  Points 

Instances  of  curvature  primitives  detected  by  the  parsing  process  are  represented 
by  a  part  of  the  tree  starting  with  a  peak  at  the  scale  at  which  the  instance  was 
first  detected.  The  exact  position  of  each  primitive,  which  gives  rise  to  knot  points 
on  the  contour,  is  computed  from  the  tree.  For  corner  primitives,  the  zero  crossing 
between  peaks  at  the  smallest  scale  gives  good  localization.  For  smooth  joins,  on  the 
other  hand,  good  localization  is  accomplished  at  the  largest  scale.  The  primitives, 
sorted  according  to  the  scale,  provide  us  with  a  multi-scale  interpretation  of  the 
contour. 

4.  Examples 

In  this  section  we  show  some  examples  of  the  multi-scaled  interpretation  of  a 
contour.  The  results  are  for  the  set  of  tools  shown  in  Figures  12  through  15. 

Figure  16  shows  how  the  curvature  primal  sketch  procedure  works  on  the 
shape  of  the  punch  shown  in  Figure  12.  Figure  16a  shows  the  positions  of  the 
local  positive  maxima  (+)  and  negative  minima  (-)  in  the  first  derivatives  (top)  and 
second  derivatives  (bottom).  Connecting  lines  arc  the  part  of  the  tree  which  was 
generated  when  an  instance  was  found.  At  the  largest  scale,  a  —  22,  two  ends  are 
found.  Then  a  dent  at  the  scale  of  11,  two  cranks  at  o  —  8,  and  another  dent  at 
o  =  5  are  detected.  Figure  16b  illustrates  the  multiple  scale  interpretation  of  the 
shape,  it  starts  out  as  a  trapezium  and  its  fine  structure  becomes  clear  as  scale  gets 
smaller. 

Similarly,  Figure  17  shows  the  curvature  primal  sketch  representation  of  the 
carving  knife  shown  in  Figure  13.  Figure  17a  is  similar  to  Figure  16a.  At  the  largest 
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Figure  16.  :i.  'I'l  ic  positions  of  the  local  positive  maxima  (+)  and  negative  minima  (-)  in 
ilic  first  derivatives  (lop)  and  second  derivatives  (bottom)  for  the  filtered  responses  of  the  punch 
(figure  12).  Connecting  lines  are  the  part  of  the  tree  which  w;is  generated  when  an  instance  was 
found,  b.  The  multiple  scale  interpretation  of  the  shape. 

scale,  again  a  =  22,  all  the  major  curvature  discontinuities  are  found.  These  are 
the  tip  of  the  blade,  the  crank  signalling  the  join  of  the  blade  and  handle,  and  the 
hand  grip  at  the  end  of  the  handle.  At  finer  scales,  smooth  joins  are  found  on  the 
blade  and  the  handle.  These  would  occur  in  different  positions  in  different  views  of 
the  knife  (see  below). 

Figure  18a  shows  the  local  extrema  of  the  filtered  responses  for  the  Warrington 
hammer  shown  in  Figure  14.  At  the  largest  scale,  the  following  discontinuities  arc 
discovered:  the  end  of  the  handle,  the  corners  defining  the  side-end  join  of  the 
handle  and  head  of  the  hammer,  the  end  signalling  the  striking  surface  on  the  head, 
the  tip  of  the  nail  puller,  and  a  smooth  join  on  the  nail  puller.  At  smaller  scales 
additional  smooth  joins  are  found,  first  on  the  head  and  then  on  the  handle.  At 
the  largest  scale,  the  description  of  the  handle  is  that  it  is  a  cylinder  (a  “worm”  in 
the  terminology  of  Blum  and  Nagel  [1976]).  At  a  smaller  scale,  a  pair  of  inflections, 
symmetric  about  the  axis  of  the  handle,  is  found.  The  corresponding  description  of 
the  handle  is  that  it  has  a  thick  body  and  a  thinner  neck.  Heide  [1984,  forthcoming] 
shows  how  to  generate  symbolic  descriptions  of  this  sort  from  smoothed  local 
symmetries  (Hollerbach  [1975]  has  shown  how  rich  symbolir  descriptions  can  be 
generated  from  an  appropriate  geometric  analysis  of  a  shape). 

Figure  19a  shows  the  filtered  responses  of  the  screwdriver  shown  in  Figure 
15.  At  the  coarsest  scale,  the  end  of  the  blade,  the  corners  defining  the  end  of 
the  handle,  and  the  pair  of  cranks  signalling  the  join  of  the  handle  and  the  blade 
are  all  discovered.  At  scale  a  —  11,  an  inflection  is  found  on  the  handle.  The 
symmetrically  placed  discontinuity,  a  crank,  is  not  found  until  the  scale  is  8.  Note 
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Figure  17.  si.  The  positions  of  t.lio  !or:tl  positive  maxima  (+)  and  negative  minima  (-)  in 
the  lirst  derivatives  (top)  and  second  derivatives  (bottom)  for  the  filtered  responses  of  the  carving 
knife  (Figure  lit).  Connecting  lines  are  the  part  of  the  tree  which  was  generated  when  sin  instance 
was  found,  b.  The  multiple  scale  interpretation  of  the  shape. 
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Figure  18.  a.  The  positions  of  the  local  positive  maxima  (  f )  anti  negative  minima  (-)  in  (he 
first  derivatives  (top)  and  second  derivatives  (bottom)  for  the  filtered  responses  of  the  Warrington 
hammer  (Figure  M).  Connecting  lines  are  the  part  or  the  tree  which  was  generated  when  an 
instance  was  found,  b.  The  multiple  scale  interpretation  of  the  shape. 

that  an  inflection  and  a  crank  arc  ambiguous  at  certain  scales,  as  noted  earlier. 

We  finally  illustrate  the  robustness  of  our  algorithm.  In  Figure  20  each  shape 
is  the  bounding  contour  of  the  same  object  as  the  screwdriver  shown  in  Figure  19 
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